Libraries

Install R packages if needed.

# Required packages
required_packages <- c(
    "rmarkdown",
    "bookdown",
    "knitr",
    "lubridate",
    "tidyverse",
    "purrr",
    "glue",
    "lubridate",
    "sf",
    "tmap",
    "leaflet",
    "leaflet.extras"
)

# Try to install packages if not installed
default_options <- options()
tryCatch(
    {
        # Disable interactivity
        options(install.packages.compile.from.source = "always")
        
        # Install package if not installed
        for (package in required_packages) {
            is_package_installed <- require(package, character.only = TRUE)
            if (!is_package_installed) {
                cat(paste0("Installing package: ", package, "\n"))
                install.packages(package)
            } else {
                cat(paste0("Package already installed: ", package, "\n"))
            }
        }
    },
    error = function(cond) {
        stop(cond)
    },
    finally = {
        options(default_options) # reset interactivity
    }
)

Load R libraries.

library(ggplot2)
library(glue)
library(leaflet)
library(leaflet.extras)
library(lubridate)
library(sf)
library(tidyverse)
library(tmap)

Data

Read data from the data folder.

ddesc <- read_csv("../../data/data.csv")
ddesc

Toronto Bikeways

Bikeways data with manually verified (Google Street View/Earth and Web Search) painted lanes and cycle tracks for Toronto, Canada

Download Link: https://github.com/rrwen/recovr-infracycle/blob/main/data/toronto-bikeways-preprocess-v1.geojson
Download Date: June 02, 2024
Data Updated: April 08, 2024
Notes: Originally downloaded on January 2023 from bikeways data at https://open.toronto.ca/dataset/cycling-network/

# Read data
toronbike_raw <- read_sf("../../data/toronto-bikeways-2024-06-02.geojson")

# Get download date
toronbike_dldate <- ddesc %>% filter(
    file == "toronto-bikeways-2024-06-02.geojson"
) %>% pull(download_date)

Map

Only the first 1000 records are shown.

tmap_mode("view")
tm_shape(toronbike_raw %>% head(1000)) +
    tm_lines(
        col = "#336699",
        border.col = "white",
        popup.vars = TRUE
    )

Data

Columns: 23
Rows: 1323

toronbike_raw %>% as_tibble

Dictionary

The data contains the following columns:

toronbike_ddict <- read_csv("../../data/toronto-bikeways-2024-06-02-datadict.csv")
toronbike_ddict

Details

print(toronbike_raw)

## Simple feature collection with 1323 features and 22 fields
## Geometry type: MULTILINESTRING
## Dimension:     XY
## Bounding box:  xmin: -79.63039 ymin: 43.58221 xmax: -79.11803 ymax: 43.85546
## Geodetic CRS:  WGS 84
## # A tibble: 1,323 × 23
##    id    street    street_from street_to road_type road_type_recode install_year
##    <chr> <chr>     <chr>       <chr>     <chr>     <chr>                   <dbl>
##  1 8     Bloor St… Parliament… Castle F… Major Ar… Arterial                 2001
##  2 17    Lake Sho… Humber Bay… Humber B… Major Ar… Arterial                 2001
##  3 18    Lake Sho… 37 M E Fle… Humber B… Major Ar… Arterial                 2001
##  4 19    Lake Sho… 50.7 M E L… 37 M E F… Major Ar… Arterial                 2001
##  5 38    Queens Q… Martin Goo… Bathurst… Collector Collector                2001
##  6 39    Davenpor… Cottingham… Macphers… Minor Ar… Arterial                 2001
##  7 40    Elizabet… College St  Gerrard … Collector Collector                2001
##  8 41    Gerrard … Yonge St    Church St Minor Ar… Arterial                 2001
##  9 42    Macphers… Davenport … Poplar P… Collector Collector                2001
## 10 43    Lake Sho… Marine Par… Palace P… Major Ar… Arterial                 2001
## # ℹ 1,313 more rows
## # ℹ 16 more variables: install_type <chr>, verify_install_year <dbl>,
## #   verify_install_date <chr>, verify_install_type <chr>,
## #   verify_install_comment <chr>, verify_upgrade1_year <dbl>,
## #   verify_upgrade1_date <chr>, verify_upgrade1_type <chr>,
## #   verify_upgrade1_comment <chr>, verify_upgrade2_year <dbl>,
## #   verify_upgrade2_date <chr>, verify_upgrade2_type <chr>, …

Files

The data files are available below:

Verified Dates

The verification dates manually entered for the cycling infrastructure data were unstructured and do not follow a structured format suitable for analysis.

Nevan Opp nevanopp@cmail.carleton.ca went through the dates in Google Sheets, interpreted them, and formatted them into structured dates, while Richard Wen richard.wen@utoronto.ca updated and fixed errors as needed.

These structured dates can then be joined back to the unstructured dates to include higher resolution temporal data to the cycling infrastructure install and upgrade dates.

Download Link: https://github.com/rrwen/recovr-eval/blob/main/data/verify-dates-2024-06-12.csv
Download Date: June 12, 2024
Data Updated: June 12, 2024
Notes: See sheet “Instructions” at https://docs.google.com/spreadsheets/d/14c9qyCr8Wmy9yJzcB6x_RNP6fU3ZAiJh7e-IE47d6D8

# Read data
vdates_raw <- read_csv("../../data/verify-dates-2024-06-12.csv")

# Get download date
vdates_dldate <- ddesc %>% filter(
    file == "verify-dates-2024-06-12.csv"
) %>% pull(download_date)

Data

Columns: 8
Rows: 298

vdates_raw

Dictionary

The data contains the following columns:

vdates_ddict <- read_csv("../../data/verify-dates-2024-06-12-datadict.csv")
vdates_ddict

Files

The data files are available below:

Cleaning

Filter Empty Types

Filter out empty install/upgrade types.

# Filter out none or na
toronbike <- toronbike_raw %>%
    filter(
        !verify_install_type %in% c("None", NA) |
        !verify_upgrade1_type %in% c("None", NA) |
        !verify_upgrade2_type %in% c("None", NA)
    )

# Display non empty types in at least one of install or upgrade
toronbike %>%
    as_tibble %>%
    select(-geometry) %>%
    select(
        id,
        verify_install_type,
        verify_upgrade1_type,
        verify_upgrade2_type
    )

Add Dates

Add cleaned post-2011 dates to verified bikeways.

# Add cleaned post-2011 dates to bikeways
toronbike <- toronbike %>%
    left_join( # clean install dates
        vdates_raw %>%
            rename_all(~str_replace(., "verify_", "clean_install_")),
        by = join_by(verify_install_date == clean_install_date_raw)
    ) %>%
    left_join( # clean upgrade1 dates
        vdates_raw %>%
            rename_all(~str_replace(., "verify_", "clean_upgrade1_")),
        by = join_by(verify_upgrade1_date == clean_upgrade1_date_raw)
    ) %>%
    left_join( # clean upgrade2 dates
        vdates_raw %>%
            rename_all(~str_replace(., "verify_", "clean_upgrade2_")),
        by = join_by(verify_upgrade2_date == clean_upgrade2_date_raw)
    )

# Display cleaned dates columns
toronbike %>%
    as_tibble %>%
    select(-geometry) %>%
    select(id, verify_install_date, verify_upgrade1_date, verify_upgrade2_date, starts_with("clean_"))

Add Quarters

Assign quarters to each bikeway date, where a value of:

1: represents November (this year) to April of next year
2: represents May to October of next year

# Add quarters to bike based on clean dates
toronbike <- toronbike %>%
    mutate(
        clean_install_quarter = case_when( # install quarter
            month(clean_install_date) %in% c(11:12, 1:4) |
            (
                month(clean_install_date_start) %in% c(11:12, 1:4) &
                ( # Nov to Dec of this year
                    month(clean_install_date_end) %in% 11:12 &
                    year(clean_install_date_end) == year(clean_install_date_start)
                ) |
                ( # Jan to Apr of this or next year
                    month(clean_install_date_end) %in% 1:4 &
                    year(clean_install_date_end) == year(clean_install_date_start) |
                    year(clean_install_date_end) == (year(clean_install_date_start) + 1)
                )
            ) ~ 2, # Nov to Apr of next year
            month(clean_install_date) %in% 5:10 |
            (
                month(clean_install_date_start) %in% 5:10 &
                month(clean_install_date_end) %in% 5:10 &
                year(clean_install_date_start) == year(clean_install_date_end)
            ) ~ 1 # May to Oct
        ),
        clean_upgrade1_quarter = case_when( # upgrade1 quarter
            month(clean_upgrade1_date) %in% c(11:12, 1:4) |
            (
                month(clean_upgrade1_date_start) %in% c(11:12, 1:4) &
                ( # Nov to Dec of this year
                    month(clean_upgrade1_date_end) %in% 11:12 &
                    year(clean_upgrade1_date_end) == year(clean_upgrade1_date_start)
                ) |
                ( # Jan to Apr of this or next year
                    month(clean_upgrade1_date_end) %in% 1:4 &
                    year(clean_upgrade1_date_end) == year(clean_upgrade1_date_start) |
                    year(clean_upgrade1_date_end) == (year(clean_upgrade1_date_start) + 1)
                )
            ) ~ 2, # Nov to Apr of next year
            month(clean_upgrade1_date) %in% 5:10 |
            (
                month(clean_upgrade1_date_start) %in% 5:10 &
                month(clean_upgrade1_date_end) %in% 5:10 &
                year(clean_upgrade1_date_start) == year(clean_upgrade1_date_end)
            ) ~ 1 # May to Oct
        ),
        clean_upgrade2_quarter = case_when( # upgrade2 quarter
            month(clean_upgrade2_date) %in% c(11:12, 1:4) |
            (
                month(clean_upgrade2_date_start) %in% c(11:12, 1:4) &
                ( # Nov to Dec of this year
                    month(clean_upgrade2_date_end) %in% 11:12 &
                    year(clean_upgrade2_date_end) == year(clean_upgrade2_date_start)
                ) |
                ( # Jan to Apr of this or next year
                    month(clean_upgrade2_date_end) %in% 1:4 &
                    year(clean_upgrade2_date_end) == year(clean_upgrade2_date_start) |
                    year(clean_upgrade2_date_end) == (year(clean_upgrade2_date_start) + 1)
                )
            ) ~ 2, # Nov to Apr of next year
            month(clean_upgrade2_date) %in% 5:10 |
            (
                month(clean_upgrade2_date_start) %in% 5:10 &
                month(clean_upgrade2_date_end) %in% 5:10 &
                year(clean_upgrade2_date_start) == year(clean_upgrade2_date_end)
            ) ~ 1 # May to Oct
        )
    )

# Display quarters
toronbike %>%
    as_tibble %>%
    select(-geometry) %>%
    select(
        id,
        clean_install_date,
        clean_install_date_start,
        clean_install_date_end,
        clean_install_quarter,
        clean_upgrade1_date,
        clean_upgrade1_date_start,
        clean_upgrade1_date_end,
        clean_upgrade1_quarter,
        clean_upgrade2_date,
        clean_upgrade2_date_start,
        clean_upgrade2_date_end,
        clean_upgrade2_quarter
    )

Exploration

Explore accuracy of Toronto bikeway data compared.

Inaccurate Install Years

Inspect all bikeways where the original installation year is not equal to the verified installation year for years 2010 to 2022.

# Filter bike for unmatched install year and add diff in years
toronbike_instyearx <- toronbike %>%
    filter(
        install_year != verify_install_year &
        !verify_install_type %in% c("None", NA) &
        verify_install_year > 2009 & verify_install_year <= 2022
    ) %>%
    mutate(
        verify_install_year_diff = verify_install_year - install_year,
        verify_install_year_diff_group = case_when(
            verify_install_year_diff <= 1 & verify_install_year_diff >= -1 ~ "±1",
            verify_install_year_diff <= 5 & verify_install_year_diff >= -5 ~ "±5",
            verify_install_year_diff > 5 | verify_install_year_diff < -5 ~ paste0(
                min(verify_install_year_diff),
                " to ",
                "-6 or 6 to ",
                max(verify_install_year_diff)
            )
        ),
        verify_install_year_diff_group = factor(
            verify_install_year_diff_group,
            levels = c(
                "±1",
                "±5",
                paste0(
                    min(verify_install_year_diff),
                    " to ",
                    "-6 or 6 to ",
                    max(verify_install_year_diff)
                )
            )
        )
    ) %>%
    relocate(
        install_year,
        verify_install_year,
        verify_install_year,
        verify_install_year_diff,
        verify_install_year_diff_group,
        install_type,
        verify_install_type,
        .after = street_to
    )

# Calc seg totals
verify_rows <- toronbike %>% nrow
all_rows <- toronbike %>%
    filter(verify_install_year > 2009 & verify_install_year <= 2022) %>%
    nrow
instyearx_rows <- toronbike_instyearx %>% nrow
instyear_rows <- all_rows - instyearx_rows
instyearx1_rows <- toronbike_instyearx %>%
    filter(verify_install_year_diff_group == "±1") %>%
    nrow
instyearx5_rows <- toronbike_instyearx %>%
    filter(verify_install_year_diff_group == "±5") %>%
    nrow
instyearxr_rows <- toronbike_instyearx %>%
    filter(
        verify_install_year_diff_group != "±5" &
        verify_install_year_diff_group != "±1"
    ) %>%
    nrow

# Calc seg perc
all_rows_perc <- round(all_rows / verify_rows * 100, 2)
instyearx_perc <- round(instyearx_rows / all_rows * 100, 2)
instyear_perc <- round(instyear_rows / all_rows * 100, 2)
instyearx1_perc <- round(instyearx1_rows / instyearx_rows * 100, 2)
instyearx5_perc <- round(instyearx5_rows / instyearx_rows * 100, 2)
instyearxr_perc <- round(instyearxr_rows / instyearx_rows * 100, 2)

Verified installs: 326 (100%)
Verified installs between 2009 and 2022: 188 of (57.67%) of 326 verified installs
Installs with correct year: 141 (75%) of 188 verified installs between 2009 and 2022
Installs with incorrect year: 47 (25%) of 188 verified installs between 2009 and 2022
Installs with incorrect year (±1): 7 (14.89%) of 47 installs with incorrect year
Installs with incorrect year (±5): 6 (12.77%) of 47 installs with incorrect year
Installs with incorrect year (>±5): 34 (72.34%) of 47 installs with incorrect year

Map

# Map bike with unmatched install years
tmap_mode("view")
toronbike_instyearx_map <- tm_basemap("CartoDB.Positron") +
    tm_shape(
        toronbike_instyearx %>%
            select(!ends_with("_comment")) %>%
            st_buffer(25),
        name = "Install with Inaccurate Year"
    ) +
    tm_polygons(
        col = "verify_install_year_diff_group",
        title = "Difference in install years",
        border.col = NULL,
        popup.vars = T,
        palette = c("green", "orange", "red")
    )

# Add fullscreen control to map
tmap_leaflet(toronbike_instyearx_map) %>%
    addFullscreenControl()

Data

# Save unmatched install year bike csv
toronbike_instyearx %>%
    mutate(geometry_wkb = st_as_text(geometry)) %>%
    select(-geometry) %>%
    write_sf("../../data/archive/toronto-bikeways-instyearx-2024-06-02.csv", na = "", append = F)

# Save unmatched bike install year geojson
toronbike_instyearx %>%
    write_sf("../../data/archive/toronto-bikeways-instyearx-2024-06-02.geojson", na = "", append = F)

# Display bike with unmatched years
toronbike_instyearx %>%
    as_tibble %>%
    select(-geometry)

No Quarters Assigned

Inspect post-2011 bikeways where segments had no quarters assigned.

# Filter bike for post-2011 and no quarter
toronbike_noquarterp2011 <- toronbike %>%
    filter( # post-2011
        verify_install_year > 2011 |
        verify_upgrade1_year > 2011 |
        verify_upgrade2_year > 2011
    ) %>%
    filter( # no quarter
        is.na(clean_install_quarter) |
        is.na(clean_upgrade1_quarter) |
        is.na(clean_upgrade2_quarter)
    ) %>%
    relocate(
        verify_install_year,
        verify_install_date,
        clean_install_date_start,
        clean_install_date_end,
        clean_install_quarter,
        verify_upgrade1_year,
        verify_upgrade1_date,
        clean_upgrade1_date_start,
        clean_upgrade1_date_end,
        clean_upgrade1_quarter,
        verify_upgrade2_year,
        verify_upgrade2_date,
        clean_upgrade2_date_start,
        clean_upgrade2_date_end,
        clean_upgrade2_quarter,
        .after = street_to
    )

# Assign base cols for map
noquarter_cols <- c(
    "id",
    "street",
    "street_from",
    "street_to"
)

# Filter post-2011 install with no quarters
toronbike_noquartinstp2011 <- toronbike_noquarterp2011 %>%
    select(
        all_of(noquarter_cols),
        starts_with("install"),
        starts_with("verify_install"),
        starts_with("clean_install")
    ) %>%
    filter(
        is.na(clean_install_quarter) &
        verify_install_year > 2011 &
        !verify_install_type %in% c("None", NA)
    )

# Filter post-2011 upgrade1 with no quarters
toronbike_noquartu1p2011 <- toronbike_noquarterp2011 %>%
    select(
        all_of(noquarter_cols),
        starts_with("upgrade1"),
        starts_with("verify_upgrade1"),
        starts_with("clean_upgrade1")
    ) %>%
    filter(
        is.na(clean_upgrade1_quarter) &
        verify_upgrade1_year > 2011 &
        !verify_upgrade1_type %in% c("None", NA)
    )

# Filter post-2011 upgrade2 with no quarters
toronbike_noquartu2p2011 <- toronbike_noquarterp2011 %>%
    select(
        all_of(noquarter_cols),
        starts_with("upgrade2"),
        starts_with("verify_upgrade2"),
        starts_with("clean_upgrade2")
    ) %>%
    filter(
        is.na(clean_upgrade2_quarter) &
        verify_upgrade2_year > 2011 &
        !verify_upgrade2_type %in% c("None", NA)
    )

# Calc seg totals
verify_rows <- toronbike %>% nrow
all_rows <- toronbike %>% filter(
        verify_install_year > 2011 |
        verify_upgrade1_year > 2011 |
        verify_upgrade2_year > 2011
    ) %>%
    nrow
noquartinst_rows <- toronbike_noquartinstp2011 %>% nrow
noquartu1_rows <- toronbike_noquartu1p2011 %>% nrow
noquartu2_rows <- toronbike_noquartu2p2011 %>% nrow

# Calc seg perc
all_rows_perc <- round(all_rows / verify_rows * 100, 2)
noquartinst_perc <- round(noquartinst_rows / all_rows * 100, 2)
noquartu1_perc <- round(noquartu1_rows / noquartinst_rows * 100, 2)
noquartu2_perc <- round(noquartu2_rows / noquartinst_rows * 100, 2)

Verified installs: 326 (100%)
Verified installs or upgrades post-2011: 233 (71.47%) of 326 verified installs
Installs with no quarters: 52 (22.32%) of 233 post-2011 verified installs or upgrades
1st upgrades with no quarters: 38 (73.08%) of 52 post-2011 no quarter installs
2nd upgrades with no quarters: 3 (5.77%) of 52 post-2011 no quarter installs

Map

# Map bike with unmatched install years
tmap_mode("view")
toronbike_noquarter_map <- tm_basemap("CartoDB.Positron") +
    tm_shape(
        toronbike_noquartinstp2011 %>%
            select(!ends_with("_comment")) %>%
            st_buffer(25),
        name = "Install (Green)"
    ) +
    tm_polygons(
        col = "green",
        border.col = "green",
        popup.vars = T
    ) +
    tm_shape(
        toronbike_noquartu1p2011 %>%
            select(!ends_with("_comment")) %>%
            st_buffer(25),
        name = "1st Upgrade (Orange)"
    ) +
    tm_polygons(
        col = "orange",
        border.col = "orange",
        popup.vars = T
    ) +
    tm_shape(
        toronbike_noquartu2p2011 %>%
            select(!ends_with("_comment")) %>%
            st_buffer(25),
        name = "2nd Upgrade (Red)"
    ) +
    tm_polygons(
        col = "red",
        border.col = "red",
        popup.vars = T
    )

# Add fullscreen control to map
tmap_leaflet(toronbike_noquarter_map) %>%
    addFullscreenControl()

Data

Install

# Save post2011 no quarter install csv
toronbike_noquartinstp2011 %>%
    mutate(geometry_wkb = st_as_text(geometry)) %>%
    select(-geometry) %>%
    write_sf("../../data/archive/toronto-bikeways-noquartinstp2011-2024-06-02.csv", na = "", append = F)

# Save post2011 no quarter install geojson
toronbike_noquartinstp2011 %>%
    write_sf("../../data/archive/toronto-bikeways-noquartinstp2011-2024-06-02.geojson", na = "", append = F)

# Display data
toronbike_noquartinstp2011 %>%
    as_tibble %>%
    select(-geometry)

1st Upgrade

# Save post2011 no quarter upgrade1 csv
toronbike_noquartu1p2011 %>%
    mutate(geometry_wkb = st_as_text(geometry)) %>%
    select(-geometry) %>%
    write_sf("../../data/archive/toronto-bikeways-noquartu1p2011-2024-06-02.csv", na = "", append = F)

# Save post2011 no quarter upgrade1 geojson
toronbike_noquartu1p2011 %>%
    write_sf("../../data/archive/toronto-bikeways-noquartu1p2011-2024-06-02.geojson", na = "", append = F)

# Display data
toronbike_noquartu1p2011 %>%
    as_tibble %>%
    select(-geometry)

2nd Upgrade

# Save post2011 no quarter upgrade2 csv
toronbike_noquartu2p2011 %>%
    mutate(geometry_wkb = st_as_text(geometry)) %>%
    select(-geometry) %>%
    write_sf("../../data/archive/toronto-bikeways-noquartu2p2011-2024-06-02.csv", na = "", append = F)

# Save post2011 no quarter upgrade2 geojson
toronbike_noquartu2p2011 %>%
    write_sf("../../data/archive/toronto-bikeways-noquartu2p2011-2024-06-02.geojson", na = "", append = F)

# Display data
toronbike_noquartu2p2011 %>%
    as_tibble %>%
    select(-geometry)

Toronto Bikeways: Data Check

R Code

Richard Wen richard.wen@utoronto.ca

Brice Batomen brice.kuimi@utoronto.ca

Linda Rothman linda.rothman@torontomu.ca

July 08, 2024

Libraries

Data

Toronto Bikeways

Map

Data

Dictionary

Details

Files

Verified Dates

Data

Dictionary

Files

Cleaning

Filter Empty Types

Add Dates

Add Quarters

Exploration

Inaccurate Install Years

Map

Data

No Quarters Assigned

Map

Data

Install

1st Upgrade

2nd Upgrade